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Abstract 

The d-dimensional A-Fleming-Viot generator acting on functions g{x), with x be- 
ing a vector of d allele frequencies, can be written as a Wright-Fisher generator 
acting on functions g with a modified random linear argument of x induced by par- 
titioning occurring in the A-Fleming-Viot process. The eigenvalues and right poly- 
nomial eigenvectors are easy to see from the representation. The two-dimensional 
process, which has a one-dimensional generator, is considered in detail. A non- 
linear equation is found for the Green's function. In a model with genie selection 
the fixation probability can be computed from an algorithm in the paper. 

An application in the infinitely-many-alleles A-Fleming-Viot process is finding 
an interesting identity for the frequency spectrum of alleles that is based on size- 
biassing. 

The moment dual process in the Fleming- Viot process is the usual A-coalescent 
tree back in time. In the Wright-Fisher representation using a different set of 
polynomials gnix) as test functions produces a dual death process which has a 
similarity to the Kingman coalescent and decreases by units of one. The eigenvalues 
of the process are analogous to the Jacobi polynomials when expressed in terms 
of gn{-c), playing the role of x". 
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1. Introduction 



The d-dimensional A-Fleming-Viot process {Xt}t>o representing frequencies 
of d types of individuals in a population has a generator £ acting on bounded 
continuous functions with second derivatives described by 



„1 d 

^9{x)= j y^Xi[g{x{l - y) + yci 

■^0 i=i 



y2 



In general A is a non-negative finite measure on [0, 1]. We take a time scale 
so that A = F is a probability measure on [0, 1]. Informally the population is 
partitioned at events of change by choosing type i G {1,2, ... ,d} to reproduce 
with probability Xi, then rescaling the population with additional offspring 
y of type i so that the frequencies are x{l — y) + yei, at rate y~'^F{dy). If 
F has a single atom at 0, then {Xt}t>o is the d-dimensional Wright-Fisher 
diffusion process with generator 



1 



92 



dxjdxi 



(2) 



The general process {Xt}t>o with generator ([T]) has a diffusive component if 
F{0) > and discontinuous sample paths from jumps where the frequencies 
are changed by adding mass y to the frequency of a type and rescaling the 
frequencies. Eventu a lly th e process becomes absorbed into one state {ej}f^^. 



Eldon and Wakeleyl ( l2006l ) introduced a model where F has a single point 
of increase in (0, 1] with a possible atom at zero as well. A natural class 
that arises from discrete models is when F has a Beta(a, /3) density, par- 
ticularly a Beta(2 — a, a) density coming from a discrete model where the 
offspring distribution tails are a symptotic to a powe r law of index a. Thi s 
g-coalescent mode l is stu died in ISchweinsberg] (|2003[ ) ; iBirkner et al.l ( 120051 ) . 
Birkner and BlathI ( 120091 ) describe the A-Fleming-Viot process and discrete 
models whose limit gives rise to the process. 

The A-coalescent is a random tree back in time which has multiple merger 
rates for a specific k edges merging while n edges in the tree of 



A 



n,= ,\\i-xr-'^,k>2. 

'0 ^ 

After coalescence there are n — k + 1 edges in the tree. 



(3) 
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This process was intr oduced by iPitmanl (119991): 



Sagitovl (I1999I ) and has 



been extensively studied (jPitmaru . l2002l : iBerestycki I . l2009l ). The coalescent 
pro cess is a momen t dual to the A-Fleming-Viot process. See, for exam- 



ple lEtheridgd ( 120121 ). There is a distin ction between an unty ped coalescent 
process and a typed process such as in lEtheridge et all (120101 ). 

There is a connection between continuous state branching proc e sses and 

the A- c oalescent. For example, seelBertoin and Le Galll ( 120031 . 120061 ) ; iBirkner et al 
( 2005 ): Berestycki. Berestycki. and Limij ( 2012a 3)- The connection is through 
the Laplace exponent 



[\e-'^y-l + qy)y-'A{dy). 
Jo 



(4) 



Bertoin and Le Galll (l2006h showed that the A-coalescent 'comes down from 
infinity' under the same condition that the continuous state branching pro- 
cess becomes extinct in finite time, that is when 



dq 



■dq < 00. 



1 i^il) 

Schweinsberg (j2000h proved earlier that coming down from infinity was equiv- 
alent to 



E 



.fc=2 



A 



nk 



< 00. 



In this paper we express the A-Fleming-Viot generator acting on functions 
as a Wright-Fisher diffusion generator where the argument of the function 
is replaced by a random linear transformation. For example if c? = 2 the 
generator acting on functions of Xi = x is specified by 



Cg{x) -- 

where A = 
identical to 



x{g{x{l-y)+y) -g{x)) + {l-x) {g{x{l-y)) -g{x)) 



F{dy) 



(5) 





F, a probability measure. A Wright-Fisher generator equation, 
is 



Cg{x) 



1 



-xil - x)E 



g"{x{l-W) + VW) 



(6) 



where W = UY, Y has distribution F, U has a density 2u, n G (0, 1), ^ is 
uniform on (0, 1), and U, V, Y are independent. If = the usual Wright- 
Fisher generator is obtained. The equation (|6]) is very suggestive of a strong 
representation between the A-Fleming-Viot and Wright-Fisher processes. 
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The dimensional generator has polynomial eigenvectors and eigenvalues 
which are analogues of those in the Wright-Fisher generator. The eigenvalues 
are 



which are equal to the A-coalescent total merger rates from n blocks. If = 2 
the polynomial eigenvectors are analogues of the Jacobi polynomials. 

The two-dimensional process is considered in detail in this paper. An 
equation is found for the stationary distribution when there is mutation. This 
leads to an interesting equation for the frequency spectrum in the infinitely- 
many-alleles A-Fleming-Viot model. Let (3{z) be the frequency spectrum, 
then z(3{z) is a probability density. Let Z he a random variable with this 
density, a random variable size-biassed with repect to Z and Z* a random 
variable size-biassed with respect to 1 — Z. Then 



where the random variables are independent of each other. The left side 
is the limit distribution of excess life in a renewal process with increments 
distributed as Z, so the equation suggests a renewal process. We do not have 
a probabilistic solution of ([7]) which would possibly lead to knowing f3{z). 

In a two-dimensional process with no mutation, and genie selection, we 
obtain a computational solution for the probability of fixation of a type that 
is analogous to that in the Wright-Fisher model. 

The moment dual process in the Fleming- Viot process is the usual A- 
coalescent back in time. In the Wright-Fisher representation using a different 
set of polynomials gn{x) as test functions produces a dual death process 
which has a similarity to the Kingman coalescent and decreases by units of 
one. The ci- dimensional version gn{x) analogous to x'^ has an expectation 
in the stationary distribution of a model with parent independent mutation 
that is similar to a Dirichlet moment 



There is even an analogue of the Ewens' sampling formula in the Poisson 
Dirichlet process of 





(7) 



E[gnix)] 



uL[uT=iiij-m[a-wy-']+9.) 



nil U7=2^a-wy-'] 
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There are many intriguing analogues between the A-Fleming-Viot pro- 
cess and the Wright-Fisher diffusion process which come from the generator 
representation. Exact calculations are likely to be difficult because of the 
jump process nature of the A-Fleming-Viot process. 

2. A Wright-Fisher generator connection 

The A-Fleming-Viot generator has an interesting connection with a Wright- 
Fisher diffusion generator that we now develop. A lemma that details how 
the generator acts on exp{^^^^ (pj^j} for </> G M'' is helpful. 

Lemma 1. Let C be the A- Fleming- Viot generator IjJ^, then 

d 



i=l j^i 

X [ [ [ e^^^^''^^^-'"^+^^'"''dv2uduF{dy). 
Jo Jo Jo 



Proof. Rearrange terms as integral expressions appearing in the generator 
acting on e 



i=l 



-Ej if>jXj+<i>i \ yu 

du 



^=l ^3 ^ Jo 

i=i ^ j ' Jo J^ 



i=l 



4=1 j^i 
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The lemma now follows by multiplying by F{dy)/y'^ and integrating. 

□ 



Theorem 1. Let C be the A-Fleming-Viot generator ([7|j, V be a uniform 
random variable on [0, 1], U a random variable on [0, 1] with density 2u, < 
u < 1 and W = YU , where Y has distribution F and V, ?7, Y are independent. 
Denote the first and second derivatives of a function g{x) by 



d 



^^^^^ = 9^-^^^^' ^-^^^ = 



Then 



Cg{x) = ^ 5^ x,{6i, - xj)ELj{x{l - W) + WVe,) 



(8) 



where expectation E is taken over V, W . An alternative to Ij^ is 



g,{x{l -W) + We,) - g,{x{l -W)) 



W 



(9) 



Proof. It is enough to check that (El) is true for g{x) = exp { J2j 'PjXj} fo'^ 
<^ G M'^. Checking: 



1 



^T,j<t>j^Ai-W)+WVei 



>jXj + (1 - )E 



eE,-<A,^,(i-w)+Wei 



i=l 



which agrees with ([8]) by Lemma 1. Taking the expectation with respect to 
V gives Q. 

□ 

The Wright-Fisher generator ([2]) is included in ([8]) when 1^ = 0. 
Corollary 1. {Xi{t)}t>o is a Markov process with generator specified by 

Cg{x) = - x)E g"{x{l - W) + WV) . (10) 
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Proof Let g{x) in ([8]) be a function of the first co-ordinate only, tlien (fTOil 
follows easily, with x = xi. 

□ 

The Laplace transform of H^is related to the Laplace exponent (jlj) by 

W possibly has an atom at 

P{W = 0) = P{Y = 0) 

and is continuous for W > with a density 

fw{w) = 2wF+{w), (11) 

where ^ 

F+H = [ y~^F{dy). 

J w 

There is a correspondence between F and the distribution of W. Given a 
random variable W with a possible atom at and a density fwiw), < w < 
1, then there exists independent random variables U, Y, where U has density 
2m, < m < 1 such that W = YU if and only if = and f{w)/w is 

decreasing in (0, 1]. Potential densities for the continuous component of W 
are proportional to the Beta (a, /3) densities with a <2 and /3 > 1. 

2.1. Mutation and selection 

Mutation can be added to the model by assuming that mutations occur at 
rate 9/2 and changes of type i to type j are made according to a transition 
matrix P. This is equivalent to mutations occurring at rate 6/2 on the dual 
A-coalescent tree. The generator ([T]) then has an additional term added of 



i=l j=l 



7 



If mutation is parent independent Opji = 6i, not depending on j, and the 
additional term simplifies to 



12) 



Etheridge et al.l (120 lOl ) study a A-Fleming-Viot process with viability se- 
lection whose generator takes the form 
.1 d 



^9{x) = [ Vx,(^(a:(l -2/) + ?/ei) -^(x)) 



Fjdy) 

y2 



.1 d 



I ^Xi{g{x{l-y) + yei)-g{x)) 



Kjjdy) 

y 



J dxi 



(13) 



i=l j=l 

To describe the measures in f|T3|) let Gi, i E [d] be the A-measures for the 
individual types, which are positive measures on [0, 1] and F be a reference 
measure such that 

F{dy)-G,{dy) 



Ki{dy) 



y 



are bounded signed measures on [0, 1]. A very weak selection model analogous 
to the Wright-Fisher model with genie selection is obtained by taking 

and letting e — > 0+. Then the corresponding sequence of generators con- 
verges to 

»i d 



■^0 i=l 



F{dy) 



y 



d 



i=l k=l 
d d 



i=l j=l 



(14) 



Etheridge et al.l (120101 ) find the dual Lambda-coalescent corresponding to 
(DEI) and([IlD. 
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2.2. Fixation probability with selection when d = 2 types 

If there are d = 2 types, no mutation, X = Xi, ai = a, a2 = 0, then the 
generator f|T^ reduces to 

C^'g^x) = - x)E g"{x{l - W) + WV) - ax{l - x)g'{x) 

Let P{x) be the probabihty that the first type fixes, starting from an initial 
frequency of x. Then P{0) = 0, -P(l) = 1, and P{x) is the solution of 

CPix) = 0. 

That is 

P"{x{l-W) + WV) =(tP'{x), (15) 
or taking the expectation with respect to V, 



-E 



P'{x{l -W) + W)- P'{x{l - W)) 



w 



aP'(x). 



(16) 



Alison Etheridge and Jay Taylor have obtained an equivalent formula to f lT6|) 
in the Beta coalescent using integration by parts, private communication 
(2008). iDer. Epstein and PlotkinI ( l201lU2012l ) study fixation probabilities in 
the A-coalescent. An interesting feature is that for some A-measures and a 
it can happen that P{x) = 1 or P{x) = for all x G (0, 1). Let a > 0, then 
they show that fixation is certain if 

log(l - y) 



a > a 



-F{dy). 



In the Kingman coalescent ( fT5l) becomes 



-P'\x) = aP'ix) 



with a solution 



Pix) 



o2a 



We provide a computational solution for P[x) when fixation or loss is not 
certain from x G (0, 1). A sequence of polynomials {/i„(x)}^q that is used 
in the proof is defined as the solutions of 

'hn{x{\ -W)^W)- hn{x{\ - W)) 



E 



W 



nhn-i{x), 



(17) 
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where the leading coefficient in is 

1 



(18) 



The argument in the expectation in f|T7|) is intrepreted ) atW = 0. 

The polynomial solutions to f|T7|l are only determined up to an arbitrary 
sequence of constant coefficients in the polynomials that are then chosen 
carefully to obtain a solution for the fixation probability. The polynomials 
hn{x) imitate and are equal if = 0. Let ho{x) = 1, and 



hn{x) = 



(Xn.rX . 



r=0 



Then from (1171) for j = — 2, . . . , 

n— 1 n 



EE 

j=0 r=jr-|-l 



E 



;i - wyw-^^^ 

so equating coefficients of on both sides, 

;i - wyw-^-^ 



n-l 

QjYiy'C TXi ^^^^^ Cliy^ ]_ J *^ 1 

j=0 



E 

r=j+l 



E 



(19) 



Given the coefficients {an-ij}]Zo of hn-i{x) the coefficients of hn{x), {anj}^=i 
are recursively determined by choosing a„„ from (fT8|) . then taking j = n — 
1, . . . , in (fT9!) . There is an arbitrary choice of a„o that needs to be made at 
this stage to progress with the recursion. 

Theorem 2. Let a < a*. The fixation probability 

" (2a)" 



P{x) = {e^'^-ir'J2 



71=1 



where {Hn{x)} are polynomials derived from 



Hn{x) 



nhn-i{i)di 
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with the constants {hn{0)} chosen so that 



nhn-i{Od^ = l. (20) 







Proof. Try a series solution 



P'{x) = B{a) J2i2arcnhn-i{x), (21) 



71=1 



where {hn{x)} satisfies ffTTI) . B{a) is a constant, and {c„} are constants not 
depending on a. Then substituting in fITB]) 

oo oo 

5];(2a)"c„(n - l)hn-2{x) = (2a) 5^(2a)V„/i„_i(a:). 

n=2 n=l 

This identity is satisfied if Ci = 1, without loss of generality, and 

1 



" (n-1)! 

Integrating in (12T]) 



, n = 2, 3, . . . 



P(a;) = i?(a) / nh^.^iOd^. 

Choosing ( l20l) to hold and knowing P(l) = 1 shows that 

i?(a) = (e--l)-\ 



□ 



Corollary 2. A computational solution for P{x) is found from evaluating 
the polynomials 



Hn{x) = Y,^. 



r 



r=l 



11 



where Hi{x) = x and the coefficients {&„,r} are defined recursively from 



r=j 



^ ( . _ Je[(1 - Wy-'W'-^\ (r + = (n + l)j&„,, 



with 



n+l 

bn+U = 1 — bn+lj 
i=2 

for n = 1,2, j = n-l,...,l. 

Proof. Relating the coefficients of Hn{x) to those of hn-i{x) 

n 

bnj ~ ~Ojn—lj—l-i j = 2, ... ,71 

3 

and 

n 

bn\ = ^ — bnj ■ 

Substituting in (fT9!) and shifting the index j — )■ j + 1 completes the proof. 



□ 



2.3. Eigenstructure of the A-Fleming-Viot process 

The generator of the A-Fleming-Viot process ([H]) with mutation term ([T2l 

d „ d 



i=l j=l 



(22) 

maps (i- dimensional polynomials into polynomials of the same degree, so the 
right eigenvectors {Pn{x)} with eigenvalues — A„ are polynomials of the same 
degree satisfying 

CPn{x) = -\nPn{x). (23) 

The index n is — 1 dimensional because of the constraint that Xli ~ ^■ 

Theorem 3. Let {A„}, {Pn{x)} he the eigenvalues and right eigenvectors 
of C, [2^] . satisfying [23^] . Denote the d — 1 eigenvalues of P which have 



12 



modulus less than 1 by {(f)k}k=\ corresponding to eigenvectors which 
of a d — 1 X d matrix R satisfying 

d 

rkiPji = (pkrkj, k = 1, . . . ,d - 1. 



are rows 



1=1 



Define a d — 1 dimensional vector ^ = Rx. Then the polynomials Pn{x) are 
polynomials in ^ whose only leading term of degree n is ^" and 



Xn = ^n{n - 1)E (1 - 



e 



d-l 



rik- 



(24) 



k=l 



Proof. The second order derivative term in C acting on x'^ is 

d 



\m-2 



miirrij — Sij)x 



,7n — Ci — e-j 



+ lower order terms 



-—mim — 1)E 
2 ^ ' 



[l-W) 



m-2 



x™" + lower order terms. 



Therefore the same term acting on with m = n is 



-nin - 1)E 
2 ^ ' 



[1 — W)^ ^ ^" + lower order terms in ^. 
The linear differential term acting on is 



6 



d 



''''' ""'JdxF 



2^2 [T.p^^ 

i=i j=i 

Q d—1 d d 



k=i 1=1 j=i 

d-l 



9 



5^(1 - (pkWe- 



k=l 



(25) 



and 0261] are enough to complete the proof of 
constructed {Pm{x)}m<n- Then take 

m:m<n 



(26) 

Suppose we have 
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where the coefficients are to be determined. 



m:in<n 



m:in<n 



for determined constants bnm- Choosing anm^m = &nm completes the con- 
struction. 



□ 



Corollary 3. The generator / fiOj) with no mutation term has eigenvalues 



Xn = Xn = ^n{n-l)E[{l-Wr-^ 



repeated {^~^^ ) times and eigenf unctions {Pnix)}n>2- 



□ 



Corollary 4. In the parent independent model of mutation the generator 
has eigevalues 



Xn = Xn = -n\in- 1)E (1 - W)' 



n-2 



(27) 



repeated ("'''^ ^) times and eigenf unctions {Pn{x)}n>i- 



Proof. The transition matrix P has rows {6i/6, . . . , Od/O). The right eigen- 
vectors of P are one vector of units with eigenvalue 1 , and d—1 other vectors 
such that Yl'i=i ^kiOi/d = 0. Thus <^fc = 0, /c = 1, . . . , d — 1 and A„ is equal to 

m 



□ 



A calculation shows that 
-n(n- 1)E 



[1-W) 



n-2 



E 

k=2 



X 



nki 



(28) 



which might be expected because the A-coalescent is a death process with 
eigenvalues the total rates away from states. A sketch of the calculation is 
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that 
1 



n{n - 1)E (1 - W) 



n-2 



-n{n — 1) 



1 rl 



^0 



(1 - uyr-'2uF{dy) 



r 



k=2 



1 - (1 - vr - ny{l - yy 

Tl 

^ 1 ^nk- 



n-l 



F{dy) 



The individual rates ([3]) can be expressed as 



>nfc 



-E 
2 



Pk{n,W) - Pk-i{n,W) 



(29) 



where 



Pk(n,w) 



n — 1 
k-1 



(1 - 



is a negative binomial probabihty of a waiting time of n trials to obtain /c 
successes, where the success probability is w. The difference 

Pk(n,w) - Pk-i{n,w) 

is not necessarily positive for all w E [0, 1]. 

In two dimensions the generator is specified by 



g"{x{l -W) + WV) + -{ei-ex)g'{x 



(30) 



The eigenvalues are 



A. 



-nUn- 1)E 



n-2 



and the eigenvectors are polynomials satisfying 

CPJx) = -XnPnix), n>l. 
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The eigenvalues and polynomials do not depend on W, V for n = 1,2. 
Writing the eigenvalue equation as 

x(l - x)E[P^{x{l -W) + VW)] + (^1 - ex)P^{x) 
-n{{n- 1)E[(1 - + e}Pn{x) = 

there is a similarity to the hypergeometric equation for the Jacobi polyno- 
mials when W = 0. 



2.4. Stationary distributions 



If the mutation matrix P is recurrent, then there is a stationary distribu- 
tion for the process with generator (!22|) . Notice that the first and second 
order moments do not depend on W. In particular, for the parent indepen- 
dent model of mutation, comparing moments with those of the Dirichlet (6) 
distribution we have that for i,j = 0,l,...,d 



E 



and E 





XiXj 



9{9 + l] 



where expectation is in the stationary distribution. 

Consider the simplest case, the stationary distribution in 2 dimensions 

when the generator is ( l30l) . Let g{x) = x""*"^, then since E 

expectation in the stationary distribution 

(n + 2){n + 1 



-E 



X{1-W)+WV) X{l-X) 



n 



-E 



Cg{X) 

x'^x{ei-ex) 



with 







or 



-E 



n 





= E 




+ E 




n + 1 







\n -1 

X{l-W) + WV] X{l-X) . (31) 



Let Z he a. random variable with the size-biassed distribution of X, Z^, a 
size-biassed Z random variable and Z* a size-biassed random variable with 
respect to 1 — Z. Then knowing that 



E 



X 



X' 



e{e + i) 



E 



X{l-X) 



6(6 + 1) 



( ISTl) can be written as 
6ii6 + l) 



E 



(VZ, 



6{6, 



E 



{vzy 



6-6i 



-E 



Z*{1-W) + WV) . (32) 
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Let B be Bernoulli random variable, independent of the other random vari- 
ables in the following equation such that 

P(B = 1) = - 

then fl32|) implies the interesting distributional identity 

VZ,='^{1-B)VZ + B{Z*{1-W) + WV). (33) 

This equation may be related to a renewal process, because the distribution 
of excess life 7f in a renewal process with increments distributed as Z satisfies 

*^oo E[Z] 

where E[Z] = Oi/e. 

The identity ( 133|) implies an identity for the densities of the random vari- 
ables in the equation that is now calculated. Let the random line L = 
Z*{1 — W) + WV. The line segment L varies from min(Z*, V) to max(Z*, V) 
as W varies. The density of the line L conditional on (Z*, V) = {z,v) is, for 
min(2;, f) < u < max{z,v), 

, . 1 ^z-u 

jL\iz,v){U) = rjw 



lim P(7t >v) = P{VZ, >v)= [ 

Jr, 



\z — v\ \z — V , 
and there is a possible atom 

P{L = z\ {z,v)) = P{W = 0). 
Splitting the region hj v < z and v > z, the unconditional density of L is 
hiu) = P{W = 0)fz*{u) 

+ / fw[- — -)fz*{z)dzdv 

Jo<v<u<z<l ^ "^^ 

+ / -^—fw(- — -)fz'{z)dzdv 

Jo<z<u<v<l ~ ^ ^V — Z/ 

Changing variables in the integral 

h{u) = PiW = 0)fz*{u) 



u pi 



Jl 

1 rl 



-zfw{i)di fz*{z)dz 

3i ? 



/ / -JwmUz'{z)dz. (34) 
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The density identity equivalent to the identity ( 15^ is therefore 

fvz,{u) = P{B = 0)fvz{u) + P{B = l)fL{u). (35) 

Note that if ^ is a random variable on [0, 1] with density fc^{y) then the 
density of VC,, where V is independent of C, and uniform on [0, 1] is 



Therefore fl35|) is equivalent to 

Illl^ = PiB = 0)1^1^ + PiB = l)Mu). (36) 
Differentiating (j36l) the density fx{u) satisfies the integral equation 

ufxin) = ^jfxiu) - IfUin), (37) 

where 

/oL is similar to ( l34l) with fz*{z) replaced by z{l — z)fx{z). A straightforward 
calculation gives that when W has no atom at zero 

fM = - r ^fw{l-\^)z{l-z)fx{z)dz 
Jq u — Z i — z 

/•I 1 y 

+ fw{l--)zil-z)fx{z)dz. 

Ju Z-U Z' 

Recalling ( ITT]) another form is 

= -£F^{l-\^)zfx{z)dz 



+ f F^(l --){!- z)fx{z)dz. 



Considering 1 — X a, second integral equation is 

[l-u)fx{u) = ^jfx{u) + -^fM- 
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If 6i = 62 then fx{.z) is symmetric about 2 = 1/2, and (157|) becomes 



{n-\)fx{u) 



F+; 1 



l-u 
1 - z 



l-u 



F+ 1- 



u 



1 - 2 



zfx{z)dz 



zfx{z)dz. 



Another approach that imitates the usual way of finding the stationary 
distribution in a diffusion process is to consider the equation 



Cg{x)fx{x){x)dx = 0, 



(38) 



where g{x) is a test function. Denote cr^(x) = a;(l — x), fi{x) = 9i — 9x — 
ax{l — x) and let 



k{x) = E (1 - Wy^g{x{l -W) + VW) 
Equation (138|) is equivalent to 



d 



d 



o {x)—k{x) + iJ.{x)—g{x) fx{x)dx = 0. 



dx'^ 



(39) 



Integrating by parts in ( l39i) and taking care with boundary conditions gives 
that 



^(^'>2d^ [^^(a;)/x(a;)] - g{x)—[fxix)fx{x)] 



dx 



dx 



[-^k{x)] [\a\x)fx{x)] - k{x)^[la\x)fx{x)] +g{xMx)fx{x) 



dx^2 



li W = then k{x) = g{x) and we can conclude that fx{x) satisfies the 
forward equation, 



An equivalent approach seems difficult when k{x) 7^ g{x) 
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A probabilistic approach 

Consider two types with total mutation rate 6 = Oi + 62- Suppose the total 
jump rate is 



A 



F{dy) 



< 00. 



/o y 

Let fi{t, x) be deterministic change between jumps, beginning at frequency 
X, which satisfies the differential equation 



dfi(t, x) 



01 - efi{t,x) 



with fi{0,x) = X. That is 

x) = e"^^*x + (1 - e"^^*)p, 

where p = 61/6. 

In what follows the precise form of fi{t,x) is not used explicitly, so for 
example, a model with mutation and selection so that 



dfi{t, x) 
Jt 



01 — Ofiit, x) — a^{t, x) (1 — /i(t, x)) 



is also possible. 

The Laplace transform of the frequency just after the first jump is 



-xt 



dt 



F{dy) 



which is equal to 
Jo 

Equation fj^Oj) can be written 



dt 



F{dy) 

y2 
(40) 



2A 



-E 



+ E 
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where T has an exponential (A) distribution. Suppose that there is a station- 
ary distribution of frequencies just after jumps. Let X° be a random variable 
with this distribution. X° has a stationary distribution in a process where 



X — 



/i(r, x) (1 - Y°) + Y° with probability /i(T, x) 
/i(T, x)(l — Y°) with probability 1 — /i(T, x) 



with y° having distribution A '^F{dy). The Laplace transform of X"^ 
satisfies 

a2 



E 



2A 



-E 



MT,x°)(i-/i(r,x°))( 



-(^[/i(r,x°)(i-w)+viy] 



-E 



-0m(T,X°) 



This is equivalent to 



E 



-(j)X° 



-E 



-E 



^(T,X°)(1-^(T,X°)). 



-(/)[M(T,x°)(i-Ty)+yvK] 



(41) 

Now consider a process where A = oo, and think of A — ?■ oo to achieve this. 
In ( l4Ti) T — 7- 0, while AT = r an exponential 1 random variable. As T — t- 0, 
fi{x, T) — )■ X, and we have a limit of ( HTl) in a model with just mutation as 







-E 



X{l-X)e 



-4'[X{l-W)+VW] 



1 

-t 

2 



()E 



-0X 



ip-x) 



which is equivalent to 



E 



Ce 



-4>x 







in the generator equation. X° is replaced by X, with a stationary distribution 
of the process with generator C. 

Turning back to the jump process when A < oo consider a renewal process 
formed from the independent exponential times Ti,T2,... between jumps. 
Let Sa = Ti + ■ ■ ■ Ta, a = 1, 2, . . .. A standard renewal theory result is that 



lim 



has a probability density 



t — ma.x{Sa < t} 
P{T > r) 

E[r] ■ 



In our context this density is identical to the exponential A density. Be- 
tween the last jump and a random time distributed with this density the 
frequency changes deterministically. Therefore we have an identity 

X=^/i(To,X°). 
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2.5. Green's function 



The Green's function, whether there is mutation or not, is obtained in a 
standard approach by solving, for 7(x), the differential equation 



C^{x) = -g{x), 7(0) = 7(1) = 0. 



(42) 



Then 



7(x) 



Equation (142 p is non-linear, equivalent to 

]-x{l - x)'E\i'{x{l -W) + VW)] + ^(^1 - 9x)i{x) = -g{x), 



or 



where 



^x{l - x)k"{x) + i(^i - Oxh'ix) = -g{x), 



(43) 



k{x) = E (1 - W)-^j{x{l -W) + VW) 
In the simplest case when 9 = 0, (143|1 becomes 



k"(x) 



x{l — x) 



Taking a standard Green's function approach, with care that k{0),k{l) are 
not zero, 



k{x) = k{0){l - x) + k{l)x 



+ 1 - X 



1—7] 



dr] + X 



V 



drj. 



If g{x) = 1, X G (0, 1) then 7(x) is the mean time to absorption at or 1 
when X{0) = x. There is a non-linear equation to solve of 

k{x) = /c(0)(l — x) + k{l)x — 2(1 — x) log(l — x) — 2xlogx. 

It is possible that 7(x) = oo if the A-coalescent does not come down from 
infinity. 
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2.6. The frequency spectrum in the infinitely-many-alleles model 

We consider the infinitely-many-alleles model as a limit from a d-allele model 
with 6i = 6/d, i = 1, . . . ,d. The limit is thought of as a limit from d points 
Xf, . . . , to points of a point process {Xi}°Zi. The 1-dimensional frequency 
spectrum /i is a non-negative measure such that for suitable functions k on 
[0,1] in the stationary distribution 



E\y2 k{Xi) = / k{x)fi{dx). 
'- .-I Jo 



i=l 

Symmetry in the d-allele model shows that 



/ k{x)n{dx) = lim dE k{X 

Jo "^^oo L 



The classical Wright-Fisher diffusion gives rise to the Poisson-Dirichlet pro- 
cess with a frequency spectrum of 

H{dx) = 9x~^{l - xY~^dx, < X < 1. 

To obtain a limit in the A Fleming- Viot process let 9i = 6/d in the generator 
( 130|) . In the identity (133|) the density of Z is dzfx^{z), < 2; < 1, by 
symmetry. Let c/ 00 in the identity f l33p . Then the identity becomes 

VZ, =^ Z*{1 -W) + WV, (44) 

where is has a measure which is the size-biassed measure of zfi{dz) with 
respect to z, and Z* similarly with respect to size-biassing of 1 — 2;. The 
constant 6 appears in the identity (jH]) because of scaling in the size-biassed 
distributions. Z^, has a measure {6 + l)z'^fi{dz) and Z* has a measure 6~^{6 + 
l)z{l — z)fi{dz), < z < 1. Let fi{dz) = f3{z)dz. The integral equations 
for the stationary distribution when there are two types imply an integral 
equation for (3{x). In view of fl57|) 

= -If'.M, (45) 

where f',]^{u) is similar to f^i{u) with fx{z) replaced by (3{z). 
In the Wright-Fisher diffusion W = and the identity (jUj) is 

VZ^ =^ Z*. (46) 
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It is straightforward to verify that if Z has density ^(1 — zf ^, then fl44p is 
satisfied. A direct solution can be found in the following way. From (H^ 

yfi{y)dy = z{\-z)fi{z), (47) 

where d is defined by 



z{1 — z)(3{z)dz 

Write (gTD as 

-^log / yPiy)dy=-Oil-zr\ 
Solving this differential equation, 

log ^ yl3{y)dy = 9 log(l - z) + A, 

for a constant A. Therefore 

[\f3{y)dy={l-zf 

J z 

because yh{y)dy = 1, and since (H7|) holds 

/3(^) = ez-\i- zy-\ 0<z<l. 

2.7. A different dual process 

The typed A-coalescent tree process is a mom ent dual in the Fleming- Viot 



process, see for example lEtheridge et al.l (120101 ). We work through a different 



type of dual process which is a death process decreasing in steps of 1. Let 
d = 2 ioT simplicity. The generator £ is specified by (|5U]) . Let {gn{x)} 
be a sequence of monic polynomials that are defined below satisfying the 
generator equation 

Cgn = ^x{l-x)Eg:{x{l-W) + VW) + ^{9^-9x)gUx) 

n\ 1 
Je(1 - Wr-'[gn-,{x) - gn{x)] + n-[e,gn^^{x) - Og^ix)] 

(48) 



24 



with go{x) = 1. Equation (HSj) is an analogue of the Wright-Fisher diffusion 
when we look at 5'„,(x) = x", with the second line chosen to mimic the Wright- 
Fisher case. Rearrange the equation to define gn{x) in terms of gn-i{x) as 



^x{l - x)Eg';{x{l -W) + VW) + ^{9, - ex)g'^{x) + A„^?„(x) 



(n - 1)E(1 - W)""-' + Oi 



gn-i[.x) 



(49) 



The polynomials {gn{x)} are well defined by (l49l) by recursively calculating 
the coefficients of x"^ in gn{x) from r = n — 1, n — 2, . . . , 0. Another way 
is to calculate a series expansion in terms of the polynomial eigenfunctions. 
Suppose that 



Denote 



gn{x) = Pn{x) + y^hnrPr{x). 

r=0 

n - 1)E{1 - W)""-^ + Oi 



An - 2 



From ( 149|) and noting that 

^Pn = —^nPn, ^gn = —^ng-n + \°5'n-l 

we have that 



n-l 



fenr[— Ar + A„]Pr.(a:) — A° bn~l,rPr 



X) 



(50) 



(51) 



r=0 



r=0 



5';(x) being a monic polynomial means that = 1, / = 0, 1, . . .. Calculating 
coefficients from f l5T]) 



A^ 



An — A; 



-bn-1, 



^n^n-1 



(An — Ar.)(An-l — A^) ■ ■ ■ (A^+l — Xr) 



The eigenfunctions {Pn{x)} also have an expansion in terms of the polyno- 
mials {gr{x)}. Let 



n-l 



Pn{x) = gn{x) + ^Cnrgr{x). 



(52) 



r=0 
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From f l5U]) 



n-l 



X P 



nr+1 



r=0 



Expressing the left side by the expansion ( 152|) and equating coefficients of gr 



The coefficients therefore are 



(Aj. — A„) ■ ■ ■ (A„_i — A„) 
Scale the equation ( HHj) by taking 



A° ■ ■ ■ A? 



An. ■ ■ • Ai 

n;=i ((j - ma - wy-^ + e,) 



9n 



Denote Beta moment analog 



n;=i((j-iMi-w^r^+^i 



so 



fi'n — ^nhn- 

Note that if X has a stationary distribution then 



E 



9n{X) 



(53) 



The polynomials {P„(x)} are analogous to the monic Jacobi polynomials 
orthogonal on the Beta {61,62) distribution with {gn{x)} analogous to {x"}. 
If ly = then they are identical in the analogy. In the Jacobi polynomial 
case ( l53l) simplifies to 



. {n- r - l)\6i(ri) {n + 6)(r) 
r\ 6i(r) {n + 6')(„,_i) ' 
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The process is not reversible, so the polynomials are not orthogonal on any 
measure unless they are the Jacobi polynomials. 
To obtain a dual generator note that 

Chn = \n[hn-l — hn] 

which is correctly set up as a dual generator equation. The dual equation is 



E 



X{0)=x 



hn{X{t)) 



E 



N{0)=n 



hN(t) {x) 



or equivalent ly 



E 



X{0)=x 



9n{X{t)) 



E 



N{0)=n 



OJr. 



■9N{t)[X) 



(54) 



(55) 



where {N{t),t > 0} is a death process with rates A„. The process {N(t),t > 
0} comes down from infinity if and only if 



< oo, 



(56) 



which implies the A-coalescent coming down from infinity because 



k=2 



SO (156|) is equivalent to 



E 

n=2 



E 

.fc=2 



^nk 



< OO, 



and 



E 

n=2 



E(*-i)(:) 

k=2 ^ ' 



^nk 



< 



E 

71=2 



E 

k=2 



nk 



< OO. 



For example if W has a Beta (a, (3) distribution then E[(l — W)"'~\ ~ Cn~°', 
where C is a constant, so if a < 1, then -^n^ < because the nth 

term is asymptotic to [C /2)n^~°' . 

The transition functions for the process {N{t),t > 0} are easily found 
from an eigenfunction analysis of the Q matrix, where qjj = —Xj and Qjj-i = 
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\j. The left and right eigenvectors and rl*^' are triangular in form with 



(fc) 



/j'^'* = 0, j > k and rl""' = 0, i < k. Explicit formulae are / 



(fc) _ Jk) 

— I u 



1 and 



(fc) 



(-l)^~^A,+i---Afc 
(Aj — Afc) ■ ■ ■ (Afc„i — Afc) 
Aj ■ ■ ■ Afc_|_i 



* (Aj — Afc) ■ ■ ■ (Afc+i — Afc) 

The transition functions are then 
P{N{t)=3\N{Q)=i) 



J < k, 
i > k. 



(57) 



k=j 



Aj+i ■ ■ • Aj 



k=j 



(Aj — Afc) • ■ ■ (Afc_i — Afc)(Afc+i — Afc) ■ ■ • (Aj — A^) 

(58) 



The distribution of N{t) given an entrance boundary at infinity is the distri- 
bution as z — 7- oo which is 



P{N{t)=3 I iV(0) = oo) =Y,e'''''rt^lf\ 

k=j 



(59) 



where 



oo > . 



well defined assuming the condition f l56|) when the coalescent comes down 
from infinity. The approach in finding the eigenfunction exp ansion f or the 
transition distribution in the Kingman coalescent is taken in Tavare 
see also iGriffithsl h9H(h ). In the Kingman coalescent A^ = k{k + 6 — l)/2, 



\jj {k + e)(^k-i) 



r 



(k) 



i\ (k + e) 



(k) 



k [i + 



i > k. 



){k) 



(60) 



and 



^(k) ^ + 

k\ ■ 
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Higher dimensions 



Let L be the ci-dimensional A-Fleming-Viot generator with mutation and 
define polynomials {^gjyx)] with g^^ix) = 1 by 

Cgn{x) = -A„(7n(a^) + -^^-n((n,-l)E[(l-iy)"»-2]+^,)^„_,^(x). (61) 

i=l 

This is an analogy with the Wright-Fisher generator acting on x". The 
polynomials are well defined by recursion on their coefficients. In a similar 
calculation to the two dimensional case there is a Dirichlet moment analogue 



nti n;ii((j-i)E[(i-w^y-^]+^o 



n;=i((j-i)E[(i-i^)^-"^]+^) 



(62) 



The dual process constructed from test functions gn{X) /¥.\jjn{X)\ has tran- 
sitions 



n — )■ n — ej at rate — — 1)E 



n 



W) 



n-2 



9). 



The dual equation is similar to flMj) . Let 

9n{x) 



then 



E 



X{0)=x 



hn{x) 



hn{X{t) 



E 



•N(Q)=n 



hN{t) {x) 



(63) 



The multitype death process has transition probabilities which are easy to 
describe from the sum of the entries |A^(t)| and 



P{N{t) =m I iV(0) =n) 



Pi\N{t)\ =m\\N 



n). 



An equation analogous to the fc- dimensional Ewens' sampling formula in the 
Poisson Dirichlet process is to let 6i = 6/d, i = 1, ... ,d, then the (labelled) 
sampling formula is 



lim d 

d—^oo 



[k] 



E[gniX 
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where n = {rii, . . . , n^, 0, . . . , 0). The samphng formula hmit is 



nW' Hi 


=1 










- i)E[(i - wy-^] 4 


-e 



Mohld (|2006[ ): iLessardI ( 120101 ) study recursive equations leading to the A- 



coalescent sampling formula. The familiar Ewens' sampling formula is ob- 
tained by taking W = 0. 



3. Acknowledgments 

This research was partly done during during a visit to the Institute for Math- 
ematical Sciences at the National University of Singapore in 2011 attending 
the program on Probability and Discrete Mathematics in Mathematical Bi- 
ology and at the Department of Statistics, University of California, Berkeley, 
as a Miller Institute Visiting Research Professor in 2012. The author thanks 
the institutions for their support and hospitality. Thanks also to many col- 
leagues for their helpful discussion on this research. 



30 



References 

Berestycki, N. 2009. Recent progress in coalescent theory. Ensaios Matem- 
aticos 16, 1-193. 

Berestycki J, Berestycki N, and Limic V. 2012a. Asymptotic sampling for- 
mulae for A-coalescents. arXiv:1201.6512V l. 

Berestycki J, Berestycki N, and Limic V. 2012b. A small-time coupling be- 
tween A-coalescents and branching processes. Preprint. 

Bertoin, J. and Le Gall, J. -F. 2003. Stochastic flows associated to coalescent 
processes. Probab Theory Related Fields 126, 249-266. 

J. Bertoin and J.-F. Le Gall 2006. Stochastic flows associated to coalescent 
processes III: Limit theorems. Illinois J. Math, 50, 147-181. 

Birkner, M., Blath, J.,Capaldo, M., Etheridge, A., Mohle, M., Schweinsberg, 
J., Wakolbinger, A. 2005. Alpha-stable branching and Beta-coalescents. 
Electron. J. Probab. 10, 303-325. 

Birkner, B. and Blath, J. 2009. Measure- Valued diffusions, general coales- 
cents and population genetic inference. Chapter 12 in: Trends in Stochastic 
Analysis, Cambridge University Press. 

Der, R., Epstein, C. L. and Plotkin, J. B. 2011. Generalized population 
models and the nature of genetic drift. Theoret. Popul. Biol. 80, 80-99. 

Der, R., Epstein, C. L. and Plotkin, J. B. 2012. The Dynamics of Neutral and 
Selected Alleles When the Offspring Distribution is Skewed. Pre-published 
in Genetics. 

Eldon, B., Wakeley, J. 2006. Coalescent processes when the distribution of 
offspring number among individuals is highly skewed. Genetics, 172, 2621- 
2633. 

Etheridge, A. 2012. Some mathematical models from population genetics. 
Ecole d'Ete de Probabilitites de Saint-Flour XXXIX-2009. Springer. 

Etheridge, A. M., Griffiths, R. C, Taylor, J. E. 2010. A coalescent dual 
process in a Moran model with genie selection, and the lambda coalescent 
limit. Theor. Popul. Biol. 78 77-92. 



31 



Griffiths, R.C. 1980. Lines of descent in the diffusion approximation of neutral 
Wright-Fisher models. Theoret. Popul. Biol. 17, 37-50. 

Lessard, S. 2010. Recurrence equations for the probability distribution 
of sample configurations in exact population genetics models. J. Appl. 
Probab. 47, 732-751. 

Mohle, M. 2006. On sampling distributions for coalescent processes with 
simultaneous multiple collisions. Bernoulli 12, 35-53. 

Mohle, M., Sagitov, S. 2001. A classification of coalescent processes for hap- 
loid exchangeable models. Ann. Probab. 20, 1547-1562. 

Pitman, J. 1999. Coalescents with multiple collisions. Ann. Probab. 27, 1870- 
1902. 

Pitman, J. 2002. Combinatorial Stochastic Processes. Ecole d'Ete de Proba- 
bilitites de Saint-Flour XXXII-2002. Springer. 

Sagitov, S. 1999. The general coalescent with asynchronous mergers of an- 
cestral lines. J. Appl. Probab. 36, 1116-1125. 

Schweinsberg, J. 2000. A necessary and sufficient condition for the A- 
coalescent to come down from infinity. Electron. Comm. Probab. 5, 1-11. 

Schweinsberg, J. 2003. Coalescent processes obtained from supercritical 
Galton- Watson processes. Stoch. Proc. Appl. 106, 107-139. 

Tavare, S. 1984. Line-of-descent and genealogical processes, and their appli- 
cation in population genetics models. Theoret. Popul. Biol. 26, 119-164. 



32 



